Vehicle control system and method of use

ABSTRACT

A system for controlling a vehicle navigating a roadway, including a perception module that generates sensor data and outputs a cost map and traffic data associated with traffic objects, a behavior planning module that receives the cost map and the traffic data from the perception module and generates planner primitives, a training module that receives the cost map and the traffic data from the perception module, receives driver input from a vehicle operator, and trains the behavior planning module, a local planning module comprising a set of task blocks that receives the cost map from the perception module and the planner primitives from the behavior planning module, selects a task block, and generates control commands using the selected task block; and a control module comprising an actuation subsystem, wherein the control module receives the control commands from the local planning module and controls the actuation subsystem.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/825,906, which claims the benefit of U.S. Provisional Application No.62/521,127, filed 16 Jun. 2107, and U.S. Provisional Application No.62/429,272, filed 2 Dec. 2016, which are all hereby incorporated intheir entirety by this reference.

TECHNICAL FIELD

This invention relates generally to the vehicle control field, and morespecifically to new and useful decision-making and control systems andmethods in the vehicle control field.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic of an embodiment of a variation of the vehiclecontrol system;

FIG. 2 is a schematic of a specific example implementation of thevehicle control system;

FIG. 3 is a schematic of an embodiment of a variation of the vehiclecontrol system;

FIG. 4 is a flow chart of a specific example of a portion of a method ofuse of a variation of the vehicle control system;

FIG. 5 is a schematic of an example of a portion of an actuationsubsystem of the vehicle control system, and flow chart of a portion ofa variation of a method of use thereof;

FIG. 6 is a schematic of an example of a portion of an actuationsubsystem of the vehicle control system, and flow chart of a portion ofa variation of a method of use thereof;

FIG. 7 is schematic of a specific example of the vehicle control systemand method of use; and

FIG. 8 is a flowchart of an example implementation of the method of useof the vehicle control system.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description of the preferred embodiments of the inventionis not intended to limit the invention to these preferred embodiments,but rather to enable any person skilled in the art to make and use thisinvention.

1. Overview

As shown in FIG. 1, the system 100 for controlling a vehicle includes aperception module 110, a behavior planning module 130 that receivesinput from the perception module 110, a local planning module 140 thatreceives input from the behavior planning module 130 and the perceptionmodule 110, and a control module 150 that receives input from the localplanning module 140 and actuates control elements of the vehicle. Thesystem 100 can optionally include a mission planning module 120 thatreceives input from the perception module and provides output to thebehavior planning module, a training module 160 that receives inputsfrom the perception module and a vehicle operator and trains othermodule(s) and/or block(s) of the system 100 based on the inputs, acommunication module 170, and/or any other suitable components.

The system 100 functions to control a vehicle during operation (e.g.,driving, loading, unloading, etc.). The system can additionally oralternatively function to collect training data, and to train adecision-making model using the collected training data. The system isoperable between several operating modes, including a manual operationmode, an autonomous operation mode (e.g., level four autonomy, levelfive autonomy, etc.), a semi-autonomous operation mode (e.g., levelthree autonomy, level two autonomy, etc.), and a teleoperation mode.However, the system can be operable between any other suitable operatingmodes. The manual operation mode preferably includes a human operatorlocated at the vehicle (e.g., within the cab of a commercial truck at adriver's seat position, in a passenger's seat position, etc.) performingactions associated with vehicle control (e.g., actuating gas and/orbrake pedals, rotating the steering wheel, selecting pre-determined taskblocks, etc.), but can additionally or alternatively include anysuitable operator, in any suitable location, performing any suitableactions to operate the vehicle. The autonomous operation mode preferablyincludes an onboard computing system receiving inputs from the sensorsubsystem 111, implementing a decision-making block at the computingsystem to select a task block based on the inputs, and executinginstructions generated by the selected task block at an actuationsubsystem to control the vehicle; however, the autonomous operation modecan include performing any other suitable actions. The teleoperationmode preferably includes a remote operator transmitting controlinstructions (e.g., behavior guidelines, directives, etc.) to the system(e.g., to a decision making block of the system) and controlling thevehicle based on the control instructions, but can additionally oralternatively include receiving any type of directive or instructionsfrom a remote operator (e.g., direct steering, throttle, and/or brakinginput signals) and controlling the vehicle in any suitable manner inresponse.

2. Benefits

There is a long-felt need in the autonomous and semi-autonomous vehiclecontrol field for the generation of high quality training data formachine learning models trained by way of supervised learning. Due tothe difficulty of identifying above-average (e.g., skilled) humandrivers, generating high quality training data from human drivers can beimpractical and costly. Also, due to the differing sensing mechanismsbetween a human driver and a vehicle control system (e.g., human visionvs. machine- or computer-vision, human depth perception vs. LIDAR,etc.), correlating human sensory inputs to sensory inputs of the vehiclecontrol system in order to validate the training data (e.g., determinefalse positives and false negatives) can be difficult, impractical,and/or costly (e.g., require human intervention, manual validation,manual labeling, etc.). Furthermore, there is a long-felt need for adeterministic control layer between a trained decision-making model(e.g., a stochastically trained model, a model trained using supervisedlearning, etc.) and the actuation of the vehicle control interfaces(e.g., to facilitate post facto auditing of autonomous vehicledecision-making and/or actions).

In view of the aforementioned and for other reasons, variants of thesystems and/or methods can afford several benefits and/or advantagesover conventional technologies and methodologies used for controllingthe operation of a vehicle, collecting training data and training adecision-making model using the training data, and/or implementing adecision-making model for controlling the operation of a vehicle.

First, variants of the system can improve the field of vehicle controltechnology by performing decision-making (e.g., task selection, behaviorselection) according to machine-learning models (e.g., statisticalmodels, stochastic models, supervised learning models) and performingvehicle operation tasks and/or actions (e.g., lane changing, gearshifting, braking, etc.) according to deterministic models (e.g.,hard-coded models, explicitly programmed sets of rules, a set of staticcomputer-implemented rules, etc.).

Second, variants of the system can provide technical solutions rooted incomputer technology (e.g., computationally calibrating modules, such asa task block, with specialized datasets in an iterative process) toovercome issues specifically arising with the computer technology (e.g.,improving accuracy of selecting task blocks based on input data from thesensor subsystem, improving selection speed, etc.). For example, a taskblock (e.g., for changing lanes at highway speeds) can be calibrated bycomparing professional truck driver outputs (e.g., from the trainingmodule, from a teleoperator) to sensor measurements of vehicleperformance.

Third, variants of the system can confer improvements to the functioningof vehicle control equipment (e.g., actuation subsystems of a vehiclecontrol system). A decision-making block optimized for achieving vehicleoperation can be used in generating task-selection instructions forcontrolling task blocks, which in turn can control the vehicle viaactuation (e.g., steering actuation subsystem, pedal actuationsubsystem, etc.) to directly operate the vehicle.

Fourth, variants of the system can generate training data (e.g., fortraining machine learning models) based on teleoperator input. Forexample, a teleoperator can select a driving task based oninterpretation of received sensor data, and the selection of theteleoperator can be implemented by the system at the vehicle andrecorded as training data.

Fifth, variants of the system can map structured input (e.g., lane linesextracted from imagery, surrounding vehicles and/or other objectsextracted from imagery, traffic data extracted from imagery and/or rangedata, etc.) to structured output (e.g., driving tasks, task blocks,etc.) in addition to and/or as an alternative to precise control outputs(e.g., steering column azimuthal angles, pedal actuation distances,etc.).

Sixth, variants of the system can enhance the predictability of vehiclecontrol and enable traceability (e.g., auditing) of decision-makingprocesses implemented using the system.

However, the system and/or method can otherwise confer any suitablebenefits.

3. System

As shown in FIG. 2, an example of the system 100 for controlling anautomotive vehicle can include: a perception module 110 including asensor subsystem 111, a lane detection block 112, a lane tracking block113, an object detection block 114, an object tracking block 115, astate estimation block 116, and a cost mapping block 117, wherein theperception module outputs a localization of the vehicle 1101, a cost mapof the area proximal the vehicle 1102, and traffic data associated withtraffic objects proximal the vehicle 1103; a mission planning module 120that receives the localization 1101 and generates a route plan 1201based on the localization 1101; a behavior planning module 130 includinga prediction block 131, a trajectory generator 132, a finite stateautomator 133, a cost map updater 134, and a decision making block 135,wherein the behavior planning module 130 receives the cost map 1102 andthe traffic data 1103 from the perception module 110 and the route plan1201 from the mission planning module 120 and generates plannerprimitives 1301 based on the cost map 1102, traffic data 1103, and routeplan 1201; a local planning module 140 including a plurality of taskblocks 141 and a degenerate block 142, wherein the local planning module140 receives the cost map 1102 from the perception module 110 and theplanner primitives 1301 from the behavior planning module 130 andgenerates control commands 1401 based on the cost map 1102 and theplanner primitives 1301; and a control module 150 including a speedcontrol block 151, a steering control block 152, and an actuationsubsystem 153, wherein the control module 150 receives the controlcommands 1401 from the local planning module 140 and controls theactuation subsystem 153 based on the control commands 1401. However, invariations, the system 100 can include any other suitable module(s)and/or block(s), and can generate any suitable output(s) whereby data istransferred among the module(s) and/or block(s). It shall be noted thatblocks within modules can, in variations, have any suitable propertiesof modules of the system 100.

System modules can include any of a: process-driven module (e.g.,equation based module, differential equation module, etc.), fuzzynetwork module, clustering module, unsupervised machine learning module(e.g., artificial neural network, association rule learning,hierarchical clustering, cluster analysis, outlier detection,convolutional neural network/CNN, etc.), supervised learning module((e.g., artificial neural network, association rule learning,hierarchical clustering, cluster analysis, outlier detection,convolutional neural network/CNN, etc.), semi-supervised learningmodule, deep learning module, and/or any other suitable moduleleveraging any other suitable machine learning method, probabilisticapproach, heuristic approach, deterministic approach, and/or anycombination thereof. The inputs and/or features (e.g., parameters usedin an equation, features used in a machine learning model, factors usedin a CNN, etc.) used in a module can be determined through a sensitivityanalysis, received from other modules (e.g., as outputs), received froma user account (e.g., from the vehicle operator, from equipmentassociated with a fleet manager of a set of vehicles, etc.),automatically retrieved (e.g., from an online database, received througha subscription to a data source, etc.), extracted from sampled sensorsignals (e.g., images, etc.), determined from a series of sensor signals(e.g., signal changes over time, signal patterns, etc.), and/orotherwise determined.

The modules are preferably universally used (e.g., the modules implementthe same models across all vehicles, fleets of vehicles, etc.), but canalternatively be specific to a driver, a vehicle, a fleet of vehicles,or otherwise differ (e.g., modules can implement models havingparameters tuned to specific drivers, vehicles, fleets, geographiclocations, etc.). Different instances of the method of use can beperformed concurrently (e.g., in parallel), asynchronously, or at anyother suitable time. Modules can be generated, executed, or calibratedevery time the system is used and/or system elements are executed (e.g.,based on up-to-date information), once, at a time interval (e.g., everyday, week, month, etc.), every time a newly-received data value differsfrom a predicted data value; and/or at any other suitable frequency.Inputs and/or outputs of the modules can be associated with any suitabletemporal indicator (e.g., daily data, averages over a period of time,etc.). Additionally, any suitable inputs for a module (e.g., thedecision-making block) can be used as inputs for another module (e.g.,the training module), and any suitable outputs of a module can be usedas inputs for another module. In an example, one or more modules and/orcombination of modules of the system can be a time series module (e.g.,where the output of a module at a first time can be used as an input toa same or different module at a second time, etc.).

In a first variation, the decision-making block of the behavior planningmodule consists essentially of a trained machine-learning model, and thetask block(s) of the local planning module each consist essentially ofan explicitly-programmed set of rules associated with the respectivetasks. The decision-making block is preferably trained by way ofsupervised learning, wherein the inputs to the model include the outputsof the perception module (e.g., an image stream, rangefinding data,location data, traffic data, etc.) and the output includes a desiredvehicle action (e.g., associated with one or more task blocks and/orcombinations of task blocks) for the vehicle to perform based on thescenario represented by the inputs. For example, the decision-makingblock can be trained to recognize emergency vehicles based on auraland/or visual patterns associated therewith and detectable by theperception module, and to pull over to the shoulder of the roadway whenappropriate in response to recognizing such an emergency vehicle.However, in this variation, the decision-making block does not generateoutputs that control the vehicle directly (e.g., command outputs,control instructions, signals for driving the actuation subsystem,etc.); instead, the task block(s) generate such outputsdeterministically (e.g., as an output of the explicitly programmed setsof rules based on the inputs from the perception module). However, infurther variations, the decision-making block and task block(s) can beotherwise suitably implemented in any suitable manner.

3.1 Perception Module

The perception module 110 of the system 100 functions to perceive theenvironment surrounding the vehicle, and output sensor data 1100indicative of features of the surrounding environment. The perceptionmodule 110 can function to directly perceive the environment (e.g., tosample imaging sensors, rangefinding sensors, etc.) as well as toperform analysis related to perception (e.g., object detection, objectclassification, image feature detection, image feature tracking, trafficdata extraction, etc.). The perception module 110 includes a sensorsubsystem 111, and can include a lane detection block 112, a lanetracking block 113, an object detection block 114, an object trackingblock 115, a state estimation block 116, and a cost mapping block 117.However, the perception module 110 can additionally or alternativelyinclude any other suitable blocks and/or subsystems for perceiving thesurrounding environment.

The perception module 110 preferably outputs sensor data 1100, which caninclude a localization of the vehicle 1101, a cost map of the areaproximal the vehicle 1102, and traffic data associated with trafficobjects proximal the vehicle 1103. However, the perception module 110can additionally or alternatively output any suitable sensor dataderived from and/or otherwise related to data gathered and/or generatedby the perception module 110.

The localization of the vehicle 1101 preferably includes the absolutegeographic location of the vehicle (e.g., a geographic location, GPScoordinates), but can additionally or alternatively include relativedistance(s) between the vehicle and objects proximal the vehicle (e.g.,roadway edges, lane edges, nearest-neighbor vehicles, traffic signage,etc.), and/or any other suitable location or positional information.

The cost map of the area proximal the vehicle 1102 preferably includes atwo-dimensional mapping of the physical space surrounding the vehicle tocost values, wherein the cost value of each point in space correspondsto an arbitrary cost value associated with the vehicle being located atthat point in space. The cost value is preferably associated with thequantitative risk of an adverse event (e.g., collision, undesirablemaneuver, violation of a traffic regulation, etc.) occurring in theevent that the vehicle were to move into and/or remain at the point inspace associated with the cost value. A high cost value (e.g., relativeto the minimum cost value in the map area) preferably corresponds to ahigh risk of an adverse event, whereas a low cost value preferablycorresponds to a low risk of an adverse event; however, in alternativeimplementations, a high cost value can correspond to a low risk of anadverse event and vice versa. A desirable trajectory through physicalspace preferably corresponds to a minimum integrated cost of thetrajectory when mapped onto the cost map; however, in alternativeimplementations, a desirable trajectory through physical space cancorrespond to a maximum integrated cost of the trajectory. The costvalue of each point in space can take on any suitable numerical value,determined from among any suitable range of values (e.g., 0 to 1,-infinity to infinity, 0 to 100, etc.).

Traffic data associated with traffic objects proximal the vehicle 1103preferably includes data related to traffic that can be used toimplement driving according to relevant laws and/or regulations. Trafficdata can include data related to the roadway. For example, traffic datacan include data indicative of the type of roadway (e.g., surfacestreet, highway, loading dock, private lot, etc.), data indicative ofallowed or disallowed vehicles for a roadway (e.g., data extracted fromsignage indicating that vehicles above a certain weight are prohibited),data indicative of traffic flow guidelines on the roadway (e.g., lanelines, words painted onto the roadway surface, signage, etc.), and anyother suitable roadway data. Traffic data can include data related toobjects on the roadway. For example, traffic data can include datarelated to nearby vehicles (e.g., number of nearby vehicles, types ofnearby vehicles, operating specifications of nearby vehicles, speedand/or trajectories of nearby vehicles, etc.), data related topedestrians (e.g., the presence of pedestrians, speed and/or trajectoryof pedestrians, etc.), data related to physical traffic flow barriers(e.g., temporary barriers, traffic cones, etc.), and any other suitabledata related to objects in the roadway. Traffic data can include anydata regarding the traffic environment that can be used to abide bylocal and/or state driving regulations. In another example, the trafficdata can include a set of positions and/or a set of trajectoriesassociated with a set of traffic objects (e.g., a neighboring vehicle, alane marking, a roadway edge, etc.) that are classified by an objectanalysis block (e.g., object detection block, object tracking block) ofthe perception module. However, traffic data can additionally oralternatively include any suitable data related to vehicular trafficand/or the vehicular traffic environs (e.g., adjacent pedestrianbyways).

The sensor subsystem 111 of the perception module 110 functions tocollect localization data and mapping data from vehicle surroundings.The sensor subsystem 111 can additionally function to collect vehicleoperation data (e.g., time series data corresponding to the state ofvehicle components during operation), and to record measurementsindicative of the driving context (e.g., ambient environment parameters,vehicle location, etc.) and provide outputs (e.g., recordedmeasurements, processed sensor data, etc.) to the decision-making block(e.g., as signals, messages, etc.) of the behavior planning module 130and any other suitable block(s) of other module(s) of the system 100.The sensor system includes at least one mapping sensor and at least onemonitoring sensor, but can additionally or alternatively include anysuitable number of any suitable sensors. The sensor subsystem 111preferably gathers the sensor data, such as image data (e.g., stillimages, video streams, compressed image sequences, etc.), range data(e.g., time-of-flight/ToF data, LIDAR data, radar data, stereocameradata, optical flow data, point cloud data, etc.), environmental data(e.g., position data, temperature data, humidity data, etc.), and anyother suitable sensor data. Inputs (e.g., gathered by the sensorsubsystem) are preferably received from one or more sensors, which canbe on-board the vehicle, removed from the vehicle (e.g., remote), or inany other suitable location. Outputs of the sensor subsystem 111 caninclude the sensor data (e.g., raw data, unprocessed digitized imagedata, etc.), as well as data derived from the sensor data, such as fuseddata (e.g., image and range data that has been combined at a processorto increase accuracy), otherwise processed data (e.g., classifiedobjects, position of object center-of-mass/CoM, object extent in three-or two-dimensional space, object trajectory, etc.), and any othersuitable data. Outputs are preferably provided to the decision-makingblock; exemplary sensor measurements provided to the decision-makingblock include visual markers on the roadway, navigation data (e.g., mapposition data, GPS data), RADAR data, orientation sensor data, ambientenvironmental data (e.g., acoustic data, temperature data, etc.), andany other suitable sensor data.

Examples of sensors and/or data sources on-board the vehicle include: aninertial measurement unit (IMU), ultrasonic sensors, a data port (e.g.,on-board diagnostic module port/OBD port), GPS sensor(s) and modules,cameras (e.g., stereoscopic cameras, single lens cameras, etc.),navigation sensors (e.g., LiDAR, radar, ToF, etc.), position sensors(e.g., actuator position sensors, LVDT sensors, etc.), encoders (e.g.,rotary encoders that measure the angular position and/or velocity ofrotary actuators of the vehicle and/or vehicle control systems), and anyother suitable on-board sensors. The aforementioned sensors of thesensor subsystem 111 can additionally or alternatively be located remotefrom the vehicle (e.g., a stationary roadside sensor, a traffic camera,etc.) and transmit sensor data to the vehicle.

Sensor data can be pre-processed by the sensor subsystem 111 and/orcomputing systems in communication with the sensor subsystem 111, priorto provision to the decision-making block and/or other modules of thevehicle control system. Pre-processing can be performed on camera data,GPS data, radar and/or other range data, and any other suitable data.For example, camera data (e.g., images) can be processed to extract lanemarkings, vehicle objects, visual landmarks (e.g., stop signs,pedestrian crossing signs, buildings, etc.), and any other suitablefeatures. In another example, GPS coordinates can be combined with a mapof the vehicle route (e.g., retrieved from a remote server, stored in anonboard database, etc.) to provide navigation data. In another example,radar signatures (e.g., point cloud data) are reduced to trajectories ofvehicles in the field of view (FoV) including the velocity and range ofthe vehicles.

Mapping sensors of the sensor subsystem 111 function to autonomously mapand/or localize the vehicle in three-dimensional space relative to thesurroundings of the vehicle. Mapping sensors can also function to detectand/or track moving objects that are proximal the vehicle (e.g., on ornear the same roadway as the vehicle). Mapping sensors can include, forexample: radar, LiDAR, cameras (e.g., stereo cameras, monocularcameras), and any other suitable mapping sensors. However, the mappingsensors can include any other suitable sensors and be otherwise suitablyarranged.

Monitoring sensors of the sensor subsystem 111 function to monitorcontrol inputs being applied to the vehicle (e.g., by a driver locatedin the vehicle, by a teleoperator, by an autonomous computing system,etc.). Monitoring sensors can include, for example: driver-facingcameras, pressure sensors and/or other force sensors that measure forceapplied to vehicle control surfaces (e.g., steering wheel, pedals,etc.), and any other suitable monitoring sensors.

The lane detection block 112 of the perception module 110 functions toextract information from the raw sensor data gathered by sensors of thesensor subsystem 111 indicative of traffic lanes on the roadway. Forexample, the lane detection block 112 can extract fiducial information(e.g., lane lines) from an image of the roadway and compute the relativespatial orientation between the vehicle and the detected lane, and theperception module no can output the relative spatial orientation astraffic data and superimpose the fiducial markings of the lane at adisplay of a remote operator interface. In another example, the lanedetection block 112 can extract a lane edge from a vibration signal ofan inertial measurement unit of the sensor subsystem 111, wherein thevibration signal is characteristic of a morphological roadway featurethat indicates a lane edge. However, the lane detection block 112 canadditionally or alternatively detect a lane in any suitable manner basedon any suitable information.

The lane tracking block 113 of the perception module 110 functions totrack the position(s) of lane(s) on the roadway, and to track the boundsof the lane for lane-keeping purposes. For example, the lane trackingblock 113 can project the continuous outline of the lane in time andspace (e.g., toward the horizon in front of the vehicle) based onindividual lane markings (e.g., discontinuous lane markings) extractedfrom images by the lane detection block 112. However, the lane trackingblock 113 can otherwise suitably track any suitable properties oflane(s) on the roadway.

The object detection block 114 of the perception module 110 functions todetect objects proximal the vehicle based on the sensor data gathered bythe sensor subsystem. Such objects can include neighboring vehicles(e.g., vehicles within one vehicle-length of the vehicle, vehicleswithin sensor range of the sensor subsystem, etc.), static objects(e.g., road signs, sound-suppression walls, center dividers, etc.), andany other physical objects nearby the vehicle. The object detectionblock 114 can detect (e.g., identify the existence of) the objects,classify (e.g., determine the type of) the objects, and perform anyother suitable analysis related to the objects.

The object tracking block 115 of the perception module 110 functions totrack the position(s) versus time of any of the object(s) detected bythe object detection block 114. For example, the object tracking blockcan determine the previous locations of a neighboring vehicle andproject the future position of the neighboring vehicle based on aphysical kinematic model and the output of the object detection block(e.g., the position and extent of the neighboring vehicle at each timepoint). However, the object tracking block can otherwise suitably trackany suitable properties of any suitable objects, in any suitable manner.

The state estimation block 116 of the perception module 110 functions todetermine the state (e.g., output) of each of the blocks (e.g., 112,113, 114, 115, 117) of the perception module 110 at any suitable time,independently of the temporal rate at which data is gathered by thesensor subsystem 111. For example, if the sensor subsystem 111 gathersdata at 20 Hz, and the output of the object tracking block is requiredby a downstream module or block at 40 Hz, the state estimation block 116can up-sample the output of the object tracking block by projecting theposition(s) of detected objects based on previous output(s) (e.g., themost recent three positions of detected objects). However, the stateestimation block can otherwise suitably estimate the state(s) of anysuitable blocks or other elements of the perception module 110.

The cost mapping block 117 of the perception module 110 functions togenerate the cost map 1102 based on output(s) of other block(s) of theperception module 110. In some variations, the cost mapping block 117can generate the cost map 1102 based on predetermined cost distributionsassociated with object types. For example, the cost distributionassociated with an object recognized as a passenger vehicle by theobject detection block can have a predetermined peak value assigned tothe physical extent of the vehicle, and a cost value associated with thespace surrounding the passenger vehicle that decreases radially (e.g.,with 1/r, 1/r², etc.) from the edge of the physical extent of thevehicle. In another example, the cost distribution associated with anobject recognized as a heavy duty vehicle (e.g., having a greaterstopping distance and lower maneuverability than a passenger vehicle)can be weighted such that the regions to the front of and/or the sidesof the heavy duty vehicle correspond to high-cost regions. In additionalor alternative variations, the cost mapping block 117 can generate thecost map 1102 based on dynamic scenarios observed by the perceptionmodule 110. For example, the perception module 110 can determine that adetected and tracked object is moving erratically, and accordinglyassign high cost values to the region(s) proximal the erratically-movingobject. However, the cost mapping block 117 can otherwise suitablygenerate the cost map 1102 based on any suitable output(s) of block(s)of the perception module 110, and/or any other suitable sensor data.

In one variation, the cost mapping block generates the cost map based onan analysis of sensor data received from the perception module. The costmap in this variation includes a two-dimensional mapping of a set ofweights to an associated set of positions on the roadway. Each weight ofthe set of weights corresponds to a risk value (e.g., a quantitativelikelihood, a categorical likelihood of a set of ordered likelihoodssuch as high, medium, and low, etc.) that an adverse event can occur inthe event that the vehicle were to be located at the associatedposition. However, the cost map can be otherwise suitably generated bythe cost mapping block in additional or alternative variations.

3.2 Mission Planning Module

The system can include a mission planning module 120, which functions todefine a route between an origin and destination and guide the vehiclealong the route during operation. The origin can be the current positionof the vehicle, the past position of the vehicle, the last position ofthe vehicle before the vehicle began moving, and/or any other suitablegeographic location of the vehicle. The destination is preferably thedesired end point of the vehicle, and can be received (e.g., from ateleoperator), generated (e.g., by the mission planning module itself),or otherwise suitably obtained.

The mission planning module 120 preferably outputs a route plan 1201.The route plan 1201 preferably includes a sequence of driving behaviors(e.g., highway entrances, highway driving, highway exits, surface streetnavigation actions, etc.) required to navigate the vehicle between theorigin and the destination. The mission planning module 120 preferablyupdates the route plan 1201 (e.g., continuously updates the vehicleposition along the route plan) based on the localization signal 1101(e.g., received by the mission planning module from the perceptionmodule); additionally or alternatively, the mission planning module 120can otherwise suitably generate and/or modify the route plan 1201 basedon any suitable information. The route plan 1201 can include graphicalindicators (e.g., arrows, street and/or exit names or numbers, etc.)that can in turn be provided at a teleoperation interface (e.g.,rendered at a heads-up display of a remote operator). However, themission planning module can additionally or alternatively provide anysuitable output in any suitable manner.

In one variation, the mission planning module receives a geographiclocation from the perception module 110 and generates a route plan basedon the geographic location and a predetermined destination. Thepredetermined destination can be entered manually by a vehicle operatorand/or automatically determined (e.g., by reading a destination tagattached to vehicle cargo), or otherwise suitably determined.

3.3 Behavior Planning Module

The behavior planning module 130 functions to plan the vehicle behaviorbased on output(s) of the perception module 110. The behavior planningmodule 130 can also function to plan the vehicle behavior based on theroute plan 1201 received from the mission planning module. The behaviorplanning module 130 can also function to plan the vehicle behavior basedon instruction(s) received from a remote operator (e.g., ateleoperator). The behavior planning module 130 can also function togenerate an error output and/or failure flag 1302 (e.g., based on areceived error output 1402 from the local planning module 140), and toprovide the error output 1302 to the mission planning module 120 and/orteleoperator 900, so that further decision making can be performed.

The behavior planning module 130 includes a decision-making block 135,and can include a prediction block 131, a trajectory generator 132, afinite state automator 133, and a cost map updater 134. The behaviorplanning module 130 and associated blocks are preferably implemented atleast in part at a local computing system (e.g., residing at thevehicle) and at least in part at a remote computing system (e.g., aremote operation system or teleoperation system communicatively linkedto the vehicle), but can additionally or alternatively be implementedentirely locally, entirely remotely, or with any suitable distributionbetween local and remote computing resources.

The behavior planning module 130 preferably receives the cost map 1102and the traffic data 1103 from the perception module 110 and the routeplan 1201 from the mission planning module 120 and generates plannerprimitives 1301 based on the cost map 1102, traffic data 1103, and routeplan 1201. In an alternative variation, the behavior planning module 130receives the cost map 1102 and traffic data 1103 from the perceptionmodule 110, and a directive from a remote operator (e.g., received atthe decision-making block 135) and generates planner primitives 1301based on the cost map 1102, traffic data 1103, and the directive. Inanother variation, the behavior planning module generates the plannerprimitives based on output of the prediction block, which includesestimated future positions of nearby traffic objects. However, thebehavior planning module can additionally or alternatively receive anysuitable inputs from any other modules of the system 100, and generateany suitable outputs in any suitable manner.

The planner primitives 1301 are preferably high-level instructions thatsemantically describe the desired vehicle behavior. For example, theplanner primitives 1301 can include instructions to stay within thecurrent lane on the roadway (e.g., a lane keeping primitive),instructions to hold the present speed (e.g., a maintain-stateprimitive), instructions to change lanes (e.g., a lane-changeprimitive), and any other suitable high-level instructions. The plannerprimitives 1301 are preferably generated by the decision-making block135 and can incorporate inputs from other block(s) of the behaviorplanning module 130 and/or can be based at least in part on the cost map11-2. traffic data 1103, and route plan 1201, either directly orindirectly. However, the planner primitives 1301 can additionally oralternatively include low-level instructions (e.g., analog signalssuitable for operating actuators, detailed instructions for operationthat include specific distances, angles, and time periods, etc.) and/orany other suitable output(s) of the behavior planning module 130.

The decision-making block 135 functions to determine a task block toexecute, based on inputs. The decision-making block can also function toselect a vehicle action from a set of available vehicle actions (e.g.,as defined by task blocks) based on received measurements. Thedecision-making block can be executed at an on-board computing system(e.g., on the vehicle), at a remote computing system (e.g., ateleoperation system), or at any other suitable location. In caseswherein the decision-making block executes at a remote computing system,sensor streams from the sensor subsystem 111 are preferably streamed tothe remote computing system in real- or near-real time. In somevariations, a first version of the decision-making block executes at avehicle computing system, and a second version of the decision-makingblock executes at a remote computing system, wherein the first andsecond versions can execute the same and/or different processes (e.g.,complimentary, different instances, etc.). In a first variation,low-level data is streamed to the remote computing system (e.g., videodata, audio data, steering actuation assembly encoder data, dataindicative of a force applied to the accelerator pedal, etc.), and thedecision-making block at the remote computing system automaticallyselects a task block based on the low-level data. In a second variation,a subset of sensor data is streamed to a teleoperation system (e.g.,video data, audio data, etc.) and the teleoperator manually selects atask block in response to the received subset of sensor data. In a thirdvariation, processed data (e.g., sampled measurement summaries,compressed images, etc.) can be sent to the teleoperation system forteleoperator monitoring, while the underlying data is used by anon-board system to control a subset of vehicle actions. The teleoperatorcan select vehicle actions based on the processed data (e.g., coveringthe same or different type of vehicle action as that determined by theon-board system), wherein the selected vehicle actions (e.g.,identifiers, instructions, etc.) can be sent to the vehicle andsubsequently performed by the on-board system.

The decision-making block is preferably implemented as an artificialneural network (e.g., a recursive neural network/RNN, CNN, a Boltzmanmachine, an auto-encoder, a deep stacking network, etc.); however, thedecision-making block can additionally or alternatively be implementedas a deterministic module (e.g., a predetermined set of rules that canbe deterministically applied to inputs to produce a task block as anoutput), a probabilistic module (e.g., a Monte-Carlo stochasticsimulator that selects a task block based on the most probable favorableoutcome of a set of inputs), or any other suitable implementation.

The decision-making block can receive various inputs from other systemcomponents and/or related entities. In a first variation, thedecision-making block receives sensor inputs from the sensor subsystem111 (e.g., sensor data as described above). In a second variation, thedecision-making block receives operator inputs. Operator inputs caninclude, for example, inputs by a teleoperator, inputs by a localoperator (e.g., the driver turning the steering wheel, depressing thebrake pedal, activating the turn signal, etc.), or any other suitableinputs by a human entity, local or remote. In a first specific example,a local operator provides inputs that bypass the decision-making blockand directly control the vehicle. In a second specific example, thelocal operator provides inputs that are mapped to an associated taskblock (e.g., using a deterministic map, a lookup table, etc.), whereinthe associated task is automatically performed by the system (e.g., byexecuting an associated task block). In a third variation, thedecision-making block receives inputs from the actuation subsysteminputs (e.g., outputs of state sensors of the actuation subsystem). Inthe third variation, the actuation subsystem can be controlled by ateleoperator and thus the inputs received by the decision-making blockcan be indirectly received from the teleoperator (e.g., for use astraining data for the decision-making block); additionally oralternatively, the actuation subsystem can be controlled in any othersuitable manner. However, the decision-making block can receive anyother suitable inputs.

The decision-making block can generate various outputs for use by othersystem components and/or related entities. The decision-making blockpreferably determines a task block to execute based on received inputs(e.g., inputs such as those described above), but can otherwise suitablygenerate and/or determine outputs. Outputs of the decision-making blockcan include, for example: direct vehicle control instructions (e.g.,voltages and/or currents to supply to the actuation subsystem), aselected task block (e.g., wherein the task block codifies a set ofinstructions to change from the current lane to a lane to the left ofthe vehicle), a set of selected task blocks (e.g., wherein the taskblocks codify a set of instructions to change lanes to the right and toengine brake), input parameters for a task block or set of task blocks(e.g., a target speed in miles-per-hour, the last-known position ofobjects in the immediate vicinity of the vehicle, etc.), any suitablecombination of the aforementioned outputs, or any other suitableoutputs.

In a first variation, determination of a task block by thedecision-making block includes selecting one of a set of task blocks(e.g., a set including an acceleration module, a deceleration module, aspeed-maintenance module, etc.). In a second variation, determining atask block includes selecting a combination of task blocks (e.g., alane-change and an acceleration module, a lane-keeping and aspeed-maintenance module, an emergency braking module andhazard-light-activation module, etc.). In a third variation, determininga task block includes generating and/or modifying a task block (e.g.,automatically generating a list of executable instructions to accomplisha vehicle operation task).

The prediction block 131 of the behavior planning module 130 functionsto predict the likelihood of occurrence of events that may affectvehicle behavior and/or desired vehicle behavior, and provide thepredicted likelihood to the decision-making block for incorporation intothe determination of decision-making block output (e.g., plannerprimitives). For example, the prediction block 131 can predict thelikelihood that a moving object (e.g., vehicle) ahead of the vehiclewill slow down, based on observing brake lights (e.g., whether or notbrake lights of the object or around the object are activated) inimagery data gathered by the sensor subsystem 111, and can provide theprediction to the decision-making block 135. The prediction block 131can predict trajectories, positions, speeds, and any other suitableproperties of any detected objects in the area surrounding the vehicle.The prediction block 131 can predict the future state of roadwaysurfaces, roadway morphology (e.g., bank angle, curve radius, etc.), andany other suitable properties of the road on which the vehicle isoperating. The prediction block 131 is preferably implemented as adeterministic model, but can additionally or alternatively includetrained models (e.g., machine-learning models trained via supervisedlearning) and any other suitable models for implementing predictionbased on input(s) from the perception module 110 and/or other systemmodules and/or blocks. The likelihood that is output by the predictionblock is preferably a quantitative likelihood that enables thestatistical expectation of the event occurrence to be incorporated intothe decision-making process as an input; however, the likelihood canadditionally or alternatively be a binary likelihood, a qualitativelikelihood (e.g., a warning), and/or be of any other suitable format.

In one variation, the prediction block estimates future positions oftraffic objects relative to the vehicle, based on computed trajectoriesof the traffic objects as well as, in some examples, other traffic data.

The trajectory generator 132 of the behavior planning module 130functions to determine the desired trajectory for the vehicle to use totraverse the physical space surrounding the vehicle at an optimal cost(e.g., minimum cost, maximum cost, etc.). The trajectory generator 131can also function to determine a set of potential trajectories that canbe used to traverse the physical space surrounding the vehicle, each ofthe set of potential trajectories having an associated integrated cost(e.g., a sum of costs associated with each point along the trajectory bythe cost map). The trajectory generator 132 preferably generates one ormore trajectories as output(s), and provides the one or moretrajectories to the decision-making block 135. The generatedtrajectories can be two-dimensional trajectories plotted from thecurrent position of the vehicle to a desired future position of thevehicle, wherein the distance between the current position and thefuture position is determined based on the speed of the vehicle and therate (e.g., frequency) at which the system module(s) and/or block(s) areexecuted. For example, a generated trajectory can correspond to the pathdesignated for the vehicle to follow over the next 5 seconds, 15seconds, 60 seconds, and any other suitable time period. Proximal intime (e.g., coincident with, contemporaneously with, within 1 second of,etc.) the end of the time period corresponding to the completion oftravel via the generated trajectory, a new trajectory can be generatedby the trajectory generator 132. However, the trajectory generator 132can additionally or alternatively generate one or more trajectories inany suitable manner.

The finite state automator 133 of the behavior planning module 130functions to determine which behaviors are allowed and disallowed giventhe environment perceived by the perception module 110 (e.g., based onsensor data from the perception module). In variations, the set ofpotential vehicle actions (e.g., lane changing, highway exiting, highwayentering, etc.) can be reduced by excluding one or more potentialvehicle actions from the set based on a determination (e.g., includingoutputs of the perception module) that the excluded potential vehicleactions are not possible. For example, the vehicle action of lanechanging can be excluded based on outputs of the lane detection and/orlane tracking blocks that indicate only a single lane is available fortraffic moving in the direction of vehicle movement. In another example,the vehicle actin of highway exiting can be excluded based on adetermination that no highway exit is present proximal (e.g., within 5vehicle-lengths of, within 10 vehicle-lengths of, etc.) the vehicle.This limited set of potential vehicle actions (e.g., finite set ofstates) is preferably provided to the decision-making block, which inturn determines which action(s) is/are appropriate. Thus, the finitestate automator 133 can act to limit the available behaviors that can bedetermined by the decision-making block and thereby avoid the selectionor other determination of unsuitable behaviors and/or actions by thebehavior planning module 130.

The cost map updater 134 of the behavior planning module 130 functionsto update the cost map 1102 based on outputs of other block(s) of thebehavior planning module 130 (e.g., 131, 132, 133, 135). Updating thecost map can include reweighting the cost associated with each point inspace based on outputs received and/or generated by block(s) and/orother portions of the behavior planning module 130. For example, ateleoperator can label a nearby vehicle as an “erratic” vehicle andtransmit the label to the behavior planning module (e.g., in the form ofa directive to avoid the vehicle), and in response the cost map updatercan increase the cost associated with the area(s) surrounding thelabeled vehicle (e.g., in real-time, in substantially real time, etc.).In another example, the finite state automator can determine that theroadway is a two-lane roadway, with a single lane corresponding to eachtraffic direction, and the cost map updater can incorporate thatdetermination into an updated cost map that reflects a high cost (e.g.,maximum cost) associated with the area of the opposing-traffic lane.However, the cost map updater 134 can additionally or alternativelyupdate the cost map in any suitable manner, based on any suitableinput(s), output(s), and/or other information.

3.4 Local Planning Module

The local planning module 140 functions to translate planning primitives1301 received from the behavior planning module 130, in combination withthe cost map 1102 received from the perception module 110, into lowlevel instructions for interpretation and execution by the controlmodule 150.

The local planning module 140 includes a plurality of task blocks 141,and can include a degenerate block 142. The local planning module 140 ispreferably located entirely at the vehicle, and is preferablyimplemented at least in part at a local computing subsystem (e.g., anapplication-specific integrated circuit, a GPU, a CPU, amicrocontroller, a mobile device residing at the vehicle, etc.).However, the local planning module 140 can be otherwise suitably locatedand/or implemented.

The local planning module 140 preferably receives the cost map 1102 fromthe perception module no and the planner primitives 1301 from thebehavior planning module 130 and generates control commands 1401 basedon the cost map 1102 and the planner primitives 1301. However, the localplanning module 140 can additionally or alternatively receive anysuitable inputs from any suitable block(s) and/or module(s) of thesystem 100, and accordingly generate any suitable outputs.

The task blocks function to perform a predefined task (e.g., bygenerating control outputs that can be used to control the actuationsubsystem). Each task block is preferably associated with a single,discrete vehicle action or task (e.g., lane keeping, left lane change,right lane change, turning right, exiting the highway, turning left,entering the highway, engine braking, friction/pedal braking, etc.), butcan additionally or alternatively be associated with multiple vehicleactions or any other suitable vehicle behaviors. Additionally oralternatively, each vehicle action can be performed by one or more taskblocks (e.g., operating concurrently, in series, based on a prior taskblock's output, etc.). Vehicle actions or tasks associated with eachmodule preferably include a set of deterministic vehicle controlalgorithms (and/or rule-based control algorithms), but can be otherwisesuitably defined and/or implemented. The set of task blocks preferablyreceive the outputs of the decision-making block, as described above, asinputs, but can otherwise receive any suitable inputs. For example, theset of task blocks can receive a selection of one of the set of taskblocks, a set of instructions, sensor outputs (e.g., motor operationparameters, wheel encoder data, images recorded by cameras of the sensorsubsystem, etc.). Alternatively, the set of task blocks can receiveinputs from a teleoperator (e.g., a single individual teleoperatingmultiple vehicles, a fly-by-wire system, etc.), a local operator (e.g.,a driver of the vehicle activating a selected task block manually), orany other suitable entity. Each task block can generate an output or setof outputs associated with the task codified by the task block; forexample, the task block can generate control instructions and/or controlsignals for control of the actuation subsystem. The outputs arepreferably signals suitable for driving components of the actuationsubsystem (e.g., having sufficient voltage, current, power, and the liketo operate components of the actuation subsystem such as motors, linearactuators, and the like), but can additionally or alternatively beinstructions to operate integrated driver components of the actuationsubsystem (e.g., an integrated motor controller/driver, an integratedpneumatic controller for a linear actuator, etc.) that are supplied tothe actuation subsystem for execution. However, the task blocks cangenerate any other suitable outputs.

In one variation, each of the task blocks includes anexplicitly-programmed set of rules. The task blocks can include only therule set (e.g., consist essentially of the rule set), or include otherrules. The rules can be deterministic, probabilistic, or have any othersuitable computation. In a specific example, each task block is adeterministic finite state machine. In a second specific example, eachtask block includes a set of deterministic equations. However, the taskblocks can be otherwise structured. The local planning module, in thisvariation, receives the cost map 1102 from the perception module 110 andthe planner primitives 1301 from the behavior planning module 130,selects a task block from a set of task blocks based on the plannerprimitives 1301, and generates low level instructions (e.g., controlcommands 1401) using the selected task block, wherein the task blockreceives the cost map 1101 as an input. However, the task blocks can beotherwise structured.

The degenerate block 142 functions to determine that an error hasoccurred at the control module, based on an output received therefrom(e.g., a failure flag). The degenerate block 142 can also function togenerate an error output and/or failure flag 1402 to provide to thebehavior planning module 130 (e.g., based on a received failure flag1501), so that the decision-making block 135 can determine a suitablecourse of action based on the error.

3.5 Control Module

The control module 150 functions to directly control the controlelements of the vehicle (e.g., throttle, steering, etc.) based on thecontrol commands 1401 received from the local planning module 140. Thecontrol module 150 can also function to perform any suitable controlaction in relation to the vehicle (e.g., to unlock the vehicle, to lockthe vehicle, to actuate any actuator of the vehicle, etc.). The controlmodule 150 can also function to generate a failure flag 1501, inresponse to the unsuccessful execution of a control instruction by oneor more elements of the actuation subsystem 153.

The control module 150 includes an actuation subsystem 153, and caninclude a speed control block 151, a steering control block 152, atransmission control block, and any other suitable control blocks (e.g.,for control of any actively controllable vehicle component).

The control module 150 preferably receives the control commands 1401from the local planning module 140 and controls the actuation subsystem153 based on the control commands 1401. However, the control module 150can additionally or alternatively receive any suitable inputs from anysuitable block(s) or module(s) of the system 100, and control theactuation subsystem 153 (and thereby the vehicle itself) in any othersuitable manner. In one variation, the control module 150 includes anaction subsystem 153 that is controlled by a microcontroller of thecontrol module based on the control commands (e.g., which areinterpreted by the microcontroller).

The speed control block 151 functions to generate a speed control signalthat is provided to the actuation subsystem 153 and results in controlof the speed of the vehicle. The speed control signal can be an analogsignal (e.g., an analog signal adapted to drive an electric motor), adigital signal (e.g., a signal adapted to drive a relay or transistor),a data signal (e.g., a signal adapted to be interpreted by amicrocontroller or specialized chip to drive an electromechanicaldevice), and/or any other suitable signal.

The steering control block 152 functions to generate a steering controlsignal that is provided to the actuation subsystem 153 and results incontrol of the steering angle (and thereby the heading, directly orindirectly) of the vehicle. The steering control signal can be an analogsignal (e.g., an analog signal adapted to drive an electric motor), adigital signal (e.g., a signal adapted to drive a relay or transistor),a data signal (e.g., a signal adapted to be interpreted by amicrocontroller or specialized chip to drive an electromechanicaldevice), and/or any other suitable signal.

The transmission control block functions to control the gearing of thetransmission of the vehicle, and to enable selection of the operatinggear (e.g., first gear, second gear, reverse, etc.) of the vehicle. Thetransmission control block can function to generate a transmissioncontrol signal that enables the actuation subsystem 153 to switchbetween vehicle gears to the chosen gear. The transmission control blockcan be used in conjunction with the speed control block (e.g., by way ofengine-braking and/or downshifting) to control vehicle speed, orotherwise suitably used.

The actuation subsystem 153 functions to actuate the control interfacesof the vehicle. The actuation subsystem can also function to execute aselected vehicle action (e.g., the instructions defined by a taskblock). Control interfaces actuated by the actuation subsystem caninclude human-oriented control interfaces, computer control interfaces,and any other suitable control interfaces. Human-oriented controlinterfaces can include, for example, a steering wheel, pedals (e.g.,clutch pedal, gas pedal, brake pedal, etc.), a shifter (e.g., shiftlever, shift paddle, etc.), and any other suitable control mechanismconfigured for operation by a human. Computer control interfaces caninclude, for example, a throttle actuation interface (e.g., acontrollable fuel flow rate regulator), a brake actuation interface(e.g., redundant brake calipers, electronic control mechanism forexisting brakes, etc.), a transmission actuation interface (e.g., anelectronically-controllable automatic transmission), and any othersuitable control interface configured to receive control instructionsfrom a computing system. The actuation subsystem preferably includes asteering actuation assembly and a pedal actuation assembly, but canadditionally or alternatively include any suitable actuation mechanism.

The actuation subsystem can receive inputs, including controlinstructions and/or operator inputs. Control instructions are preferablyreceived from control blocks (e.g., the speed control, steering control,and/or transmission control blocks), and can include voltage levels,current levels, time-varying signals, constant signals, trigger- and/orevent-based signals, and any other suitable instructions and/orparameters that can effect an output of the actuation subsystem (e.g.,an actuation of a control interface by the actuation subsystem).Operator inputs are preferably received from an operator (e.g., a driverin the vehicle, a teleoperator, etc.), and can include a human movingthe steering wheel by hand, a human depressing a pedal manually, atransmission from a teleoperator including instructions to actuate aportion of the actuation subsystem, and any other suitable inputs froman operator.

The actuation subsystem can provide outputs, including state dataindicative of the state of actuators of the actuation subsystem. Forexample, the actuation subsystem can provide one or more signals withamplitude values proportional to a force applied by actuators to controlinterfaces of the vehicle (e.g., the torque applied by a motor of asteering wheel actuator of the actuation subsystem, the force applied bya pedal actuator of the actuation subsystem, etc.). In another example,the actuation subsystem can provide outputs indicative of force appliedby a human to control interfaces (e.g., torque applied to a steeringwheel by a human operator, force applied to a pedal by a human operator,etc.). These outputs can transition the system between operation modes(e.g., switch system operation from an autonomous operation mode to amanual operation mode), select a task block (e.g., associated with theoutput type, value, etc.), or be otherwise used. In related examples,the actuation subsystem can provide outputs indicative of whether theactuation subsystem (or portions thereof) are powered “on” or “off”(e.g., an indicator LED that emits light when the actuation subsystem ispowered “on”), whether actuators of the actuation subsystem aremechanically engaged with or disengaged from vehicle control interfaces,and any other suitable states and/or configurations of the actuationsubsystems and/or portions thereof.

As shown in FIG. 5, a specific example of the steering actuationassembly of the actuation subsystem includes a drive motor, a torquetransfer mechanism, and an encoder. The drive motor functions togenerate a torque that can be used to mechanically rotate the rotatingportion of the steering column. The torque transfer mechanism includes adrive gear and a driven gear, wherein the drive gear receives the torquegenerated by the drive motor and transfers it to driven gear, which isfixed to the steering wheel and is rotated by the drive gear, thusrotating the steering wheel. The encoder functions to monitor the outputof the steering actuation assembly (e.g., the angular position of thedrive motor shaft, the angular position of the steering wheel, the rateof rotation of the drive motor and/or steering wheel, etc.). However,the steering actuation assembly can have any other suitable componentsthat have any suitable functionality directed to steering columnactuation.

As shown in FIG. 6, a specific example of the translation actuationassembly of the actuation subsystem includes an actuator and a forcetransfer mechanism, and can optionally include a release mechanism. Theactuator functions to displace the vehicle translation control mechanism(e.g., pedal, throttle) in order to control the vehicle componentassociated with the pedal (e.g., to brake the vehicle, to accelerate thevehicle, to shift the gears of the vehicle, etc.). The force transfermechanism functions to transmit force applied to the force transfermechanism (e.g., by a human operator, by the actuator of the pedalactuation assembly, etc.) to a pedal. The force transfer mechanismpreferably includes a monitoring sensor (e.g., a force sensor) thatcommunicates the force applied to the pedal (e.g., by the actuator, by ahuman operator) to a computing system (e.g., in a similar manner asother components of the sensor subsystem). The release mechanismfunctions to physically transition the pedal actuation assembly from theengaged configuration (e.g., in actuatable physical contact with pedalsof the vehicle) to the disengaged configuration (e.g., out of physicalcontact with the pedals of the vehicle); the release mechanism can beactuated by a human driver (e.g., by way of a release switch), a remoteoperator, an autonomous control system, or any other suitable entity.However, the pedal actuation assembly can have any other suitablecomponents that have any suitable functionality directed to pedalactuation.

3.6 Training Module

As shown in FIG. 4, the training module 160 functions to generatetraining data usable to train models implemented by the decision-makingblock. The training module can additionally function to verifyfalse-positive outputs and false-negative outputs of the decision-makingblock, and to train the decision-making block (e.g., modify parameters,properties, and other features of the decision-making block to reducethe number of false-positive and/or false-negative outputs). Inputs tothe training module can include driver behavior (e.g., actions of alocal operator, actions of a teleoperator), sensor data (e.g., receivedfrom the sensor subsystem), and any other suitable inputs from any othersuitable components of the system or related entities. The trainingmodule can also receive the output of the decision-making block (e.g., aselected task block, a set of weight parameters for provision to aselected task block, etc.). The output of the training module preferablyincludes training data that can be utilized to modify a model or modelsimplemented by the decision-making block. In a first variation, thetraining module correlates sensor data recorded on-board the vehicle(e.g., by the sensor subsystem) and driver behavior, in order to comparedriver behavior in response to driving conditions with outputs of thedecision-making block in response to sensor data. Such correlation caninclude determining a false negative (e.g., a decision-making blockdecision that did not recognize an event that led to an intervention bythe driver), determining a false positive (e.g., the decision-makingblock determined that a driver intervention was necessary and the driverdid not intervene), or making any other suitable determination.

In one variation, the training module 160 receives the cost map 1102 andthe traffic data 1103 from the perception module 110, and receivesdriver input (e.g., direct steering input from a local driver as shownin FIG. 3, remote speed input from a connected teleoperator, etc.) froma vehicle operator, and trains the behavior planning module based on thedriver input, the cost map, and the traffic data. In an example, thedriver input can include preemptively slowing the vehicle to produce agreater distance between the vehicle and a second vehicle driving aheadof the vehicle, in response to the vehicle operator noting erraticacceleration and deceleration behavior associated with the secondvehicle. In this example, the training module can associate the statesof the cost map 1102 and the traffic data 1103, extracted from the timeperiod that the vehicle operator noted this behavior and provided thedriver input, with the driver input; the training module cansubsequently train the decision-making block based on these associatedstates and behaviors, and other similarly collected and associatedstates and behaviors, such that the decision-making block canautomatically preemptively slow the vehicle in response to similarscenarios (e.g., implement supervised learning). However, the trainingmodule 160 can otherwise suitably train the decision-making block and/ormodule(s)/block(s) of the system 100.

3.7 Communication Module

The system can optionally include a communication module 170, whichfunctions to communicatively couple the vehicle control system to aremote computing system (e.g., a teleoperation system). Thecommunication module preferably includes a two-way wireless data link(e.g., 3G radio, 4G radio, 5G radios), but can additionally oralternatively include any other suitable data communication mechanism.The communication module is preferably integrated into the vehicle, butcan additionally or alternatively be integrated into a mobile deviceassociated with the vehicle and/or an occupant of the vehicle, travelingwith the vehicle, or any other suitable mobile device. The communicationmodule can receive various inputs, including transmissions sent to thevehicle (e.g., teleoperating instructions, GPS transponder or satellitesignals, etc.), data to be transmitted away from the vehicle (e.g.,sensor data, communications from a local operator designated for anentity remote from the vehicle, etc.), or any other suitable inputs. Thecommunication module preferably transmits messages as outputs; suchmessages can include communications (e.g., radio communications),monitoring data (e.g., data collected by monitoring sensors of thesensor subsystem, data indicative of vehicular state or behavior, etc.),and any other suitable message data. The communication module canadditionally or alternatively generate and/or provide other suitableoutputs.

The communication module can include a remote teleoperation interface171, which can include a display and a set of inputs. In one variation,the sensor data 1100 is transmitted to the remote teleoperationinterface 171 and rendered at the display to a remote vehicle operator.In this variation, the system can receive driver input from the remotevehicle operator at the set of inputs (e.g., a connected replicasteering wheel, an adapted game console controller, a mouse, a connectedreplica gas/brake/clutch pedal, etc.). In a related variation, thedisplay is configured to visually simulate the interior of a commercialtruck cabin (e.g., a semi-truck cabin, a semi-tractor cabin, a lorrycabin, etc.). In another related variation, the set of inputs includes asteering wheel input, a gas pedal input, a brake pedal input, and atransmission input, wherein each of the aforementioned inputs areconfigured as distinct inputs. However, the remote teleoperationinterface 171 can be otherwise suitably configured.

4. Method of System Use

As shown in FIG. 8, the method 200 includes: sampling sensor data S210;receiving a directive, wherein the directive is based on an analysis ofthe sensor data S220; generating a planner primitive S230; selecting atask block based on the planner primitive S240; controlling the vehiclebased on the selected task block in combination with the sensor dataS250; and transferring planning authority among operating entities S260.The method 200 can optionally include transmitting the sensor dataoff-vehicle S215; rendering the sensor data to a vehicle operator S217;and training a planning module based on the sensor data in combinationwith a directive S270.

The method 200 is preferably implemented at, by, and/or by use of asystem substantially identical to the system 100 described above inSection 3. However, the method 200 can additionally or alternatively beimplemented at, by, and/or by use of any suitable system forautonomously, semi-autonomously, or manually controlling a vehicle.

Block S210 includes: sampling sensor data, which functions to gathersensor data from one or more sensors of the vehicle control system. Thesensors preferably include image sensors, range finding sensors, andother sensors substantially as described above in Section 1, but canadditionally or alternatively include any suitable sensors. In a firstvariation, the sensor data includes an image stream (e.g., a sequence ofimagers, a video, etc.), a localization signal (e.g., a GPS location),and operational data (e.g., vehicle speed, vehicle fuel or energy level,vehicle internal temperature, external environmental temperature, tireinflation pressures for each tire, etc.).

In a first variation, Block S210 includes continuously sampling sensordata (e.g., at a sensor subsystem of the vehicle) that includes an imagestream, a localization signal, and operational data. The sensor data caninclude any of the data described above in relation to the sensor data1100 of the system 100, and/or any other suitable sensor data.Operational data can include any data related to operation of thevehicle, such as the current vehicle speed, current vehicle steeringangle, current throttle status, instantaneous estimated vehicle range,and any other suitable data.

The method 200 can optionally include Block S215, which includes:transmitting the sensor data off-vehicle, which functions to provide thesensor data to an operating entity of the vehicle that is also locatedoff-vehicle (e.g., a remote teleoperator). Block S215 can also functionto log the sensor data collected during vehicle operation for subsequentanalysis (e.g., training via Block S270, record collection, etc.). In afirst variation, Block S215 includes transmitting the image stream, thelocalization data, and the operational data to a remote teleoperationinterface (e.g., similar to the remote teleoperation database 171)associated with a teleoperator.

The method can optionally include Block S217, which includes: renderingthe sensor data to a vehicle operator, which functions to providevehicle telemetry to a vehicle operator. In one variation Block S217includes rendering the sensor data (e.g., operational data, trafficdata, etc.) at the remote teleoperation interface, wherein the remoteteleoperation interface is configured to visually simulate the interiorof a commercial trucking cabin.

Block S220 includes: receiving a directive, wherein the directive isbased on an analysis of the sensor data, which functions to interpretthe sensor data and provide a course of action for the vehicle based onthe interpretation of the sensor data. The directive is preferably asemantic instruction (e.g., a directive to change lanes, a directive toexit a highway, a directive to slow and stop at the side of the roadwaywhen sufficient space is available, etc.), but can additionally oralternatively include a binary instruction (e.g., emergency stop vs.continue operating), a direct control input (e.g., movement of thesteering wheel, actuation of the brake or gas pedal, changing of thetransmission gear, etc.), an indirect control input (e.g., movement of aremotely-connected steering-wheel controller, actuation of aremotely-connected pedal controller, etc.), and/or any other suitableinstruction or similar input related to vehicle operation.

In one variation, Block S220 includes receiving, at a behavior planningmodule of the vehicle, a first directive from the teleoperator by way ofthe remote teleoperation interface, wherein the behavior planning moduleconsists essentially of a trained machine-learning module. In a relatedvariation, Block S220 includes receiving a second directive from theteleoperator, in response to the vehicle entering a geographic regionhaving a predetermined characteristic. In examples, the geographicregion having the predetermined characteristic can include an end regionof a highway on-ramp (e.g., the teleoperator can transmit a directive toenter fully-autonomous mode after reaching the end of the on-ramp).

Block S230 includes: generating a planner primitive, which functions toprovide a high-level description (e.g., a semantic description) of thespecific actions to be taken in navigating the vehicle according to thedirective. Block S230 can include generating the planner primitive basedon a directive. For example, the directive can include a directive tochange lanes, and Block S230 can include generating a planner primitiveto enter a lane-change task block and a set of criteria that should bemet before initiating the lane change (e.g., an integrated costthreshold based on the cost map, a timeout limit, etc.). Block S230 caninclude automatically generating the planner primitive. For example,Block S230 can include automatically generating a planner primitive at abehavior planning module of the vehicle based on the received sensordata (e.g., operating substantially and/or completely autonomously).

Block S240 includes: selecting a task block based on the plannerprimitive, which functions to choose a task block (e.g., representativeof a desired vehicle action) based on the instruction associated withthe planner primitive (e.g., instructions to perform an action and/orset or series of actions). Block S240 can be performed in response to adirective, and/or automatically, or otherwise suitably performed.

In one variation, Block S240 includes selecting, at a local planningmodule of the vehicle, a task block based on the planner primitive,wherein the task block consists essentially of an explicitly programmedset of rules.

Block S250 includes: controlling the vehicle based on the selected taskblock in combination with the sensor data, which functions to providelow level instructions to actuators of the vehicle control system tophysically control the vehicle, based on sensor data and theinstructions codified by the selected task block.

Block S260 includes: transferring planning authority among operatingentities, which functions to hand over vehicle control between variousentities that can exert control over vehicle operation. Such operatingentities can include a computing system (e.g., a behavior planningmodule at the vehicle, a behavior planning module remote from thevehicle), a remote vehicle operator (e.g., a teleoperator), and in somecases a local operator (e.g., a driver located in the vehicle), as wellas any other suitable human and/or machine operator of the vehicle.Planning authority preferably includes decision making (e.g., such asdecision making performed by a decision-making block of the vehiclecontrol system). Transferred planning authority can, in some variations,include direct control over actuation elements (e.g., a steeringactuator, a pedal actuator), but can alternatively include onlydecision-making control wherein direct control is performed via systemsresiding at the vehicle (e.g., a local planning module, a controlmodule, etc.).

In one variation, Block S260 can be performed based on receiving adirective (e.g., an instance or example of Block S220); for example, thedirective can include an instruction to transfer planning authority froma behavior planning module residing at the vehicle to a teleoperatorlocated remotely, and Block S260 can accordingly include transferringplanning authority to the teleoperator from the behavior planningmodule. The directive can be received from (e.g., manually triggered by)an operator (e.g., teleoperator, local operator in the vehicle), or fromany other suitable source. In another example, Block S260 can includetransferring planning authority to the behavior planning module of thevehicle, in response to receiving the second directive, wherein thesecond directive is received from the teleoperator, in response to thevehicle entering a geographic region having a predeterminedcharacteristic.

In another variation, Block S260 can be performed automatically, inresponse to a predetermined condition being met. For example, Block S260can include transferring planning authority to a teleoperator inresponse to the vehicle reaching a predetermined geographic location(e.g., a highway exit along the route determined by the mission planningmodule, a highway off-ramp, etc.). In another example, Block S260 caninclude transferring planning authority from the teleoperator to thebehavior planning module (e.g., residing at the vehicle) in response tothe vehicle entering a stable speed condition on a highway after mergingfrom an on-ramp thereof. In another example, Block S260 can includetransferring planning authority to an operator (e.g., local operatorresiding in the vehicle, teleoperator, etc.) in response to anemergency-takeover switch or button being activated by the localoperator (e.g., receiving a directive from a local operator) or anemergency event (e.g., failure event) detected from the sensor data.

The method can include Block S270 which includes: training a modulebased on the sensor data in combination with a directive, whichfunctions to train a machine-learning module of the vehicle controlsystem. Training is preferably performed via supervised learning, butcan additionally or alternatively be performed via unsupervised learningand/or any one or more of the machine-learning techniques describedabove in Section 3. In a specific example, Block S270 can includetraining a behavior planning module (e.g., a decision-making block ofthe behavior planning module) based on direct control inputs provided atan actuation subsystem of the vehicle by a local operator (e.g., adriver located in the vehicle) in combination with sensor data (e.g.,image and rangefinding data) to train the behavior planning module totake the same actions with regard to controlling the vehicle as thelocal operator does when presented with a traffic situationcorresponding to the sensor data.

In a first example of the method 200, as shown in FIG. 7, thedecision-making block of the behavior planning module (e.g., on boardthe vehicle, remote from the vehicle, etc.) can select a vehicle actionand present the selected action to a teleoperator. The vehicle actioncan be subsequently performed in response to teleoperator approval ofthe selected action. Additionally or alternatively, the teleoperator canutilize a software switch that transmits a signal to the vehicle totransition out of the autonomous mode after the teleoperator has assumedcontrol of the vehicle.

In a second example of vehicle control system use (e.g., a method of useof the system), all or some of the measurements by the sensor subsystem(e.g., a low-resolution sensor stream) can be sent to a remoteteleoperator, which monitors the sensor streams and can (in somevariants) override and/or preempt the decision-making block (e.g., incases where the decision-making block has already determined or isdetermining a task block) by selecting a vehicle action (e.g., behavior)from a set of available vehicle actions (e.g., codified by task blocks).The decision-making block can be determined using the first example ofthe method, or be otherwise determined. The selected vehicle action(e.g., instructions or identifier thereof) can then be transmitted tothe vehicle, wherein the vehicle performs the teleoperator-selectedvehicle action (e.g., using the actuation subsystem). In cases where thedecision-making block-selected vehicle behavior (e.g., task) conflictswith the teleoperator-selected vehicle behavior, theteleoperator-selected vehicle behavior is preferably given preference(e.g., execution preference), but the decision-making block-selectedvehicle behavior can alternatively be given preference.

In another example of the method of use, a local driver (e.g., a humanvehicle operator) can be located within the vehicle (e.g., in a driver'sseat position of a vehicle cabin) and have the same operating authorityand capabilities as a teleoperator. The local driver's commandspreferably supersede the teleoperator's commands, which supersede thedecision-making block commands, but the commands can be prioritized inany other suitable order.

In other examples of the method of use, as shown in FIGS. 2 and 5, thetraining module can monitor the teleoperator and/or local driverinstructions (e.g., selections of task blocks by the teleoperator) andthe underlying data (e.g., sensor data collected by the sensor subsystemover a time period preceding the teleoperator selection), and train thedecision-making block based on the teleoperator selections andunderlying data (e.g., using an artificial neural network, a stochasticmachine-learning model, etc.). The trained (e.g., updated, modified,etc.) decision-making block can subsequently be supplied to the vehicle(e.g., at a predetermined frequency, when an update condition is met,etc.), and the training process can be repeated. Selections by a localdriver can be treated similarly to the teleoperator instructions by thetraining module, but can be otherwise treated.

In another specific example of system use, the sensor subsystem collectssensor data as inputs and provides the sensor data to the behaviorplanning module as outputs. The sensor data includes image datacollected from one or more outward-facing cameras of the vehicle, aswell as point cloud data form one or more rangefinding sensors of thevehicle. The decision-making block processes the sensor data accordingto a set of rules (e.g., implemented as a convolutional neural network),and produces a task selection as an output. The task selection caninclude one or more vehicle operation tasks (e.g., slow down, changelanes, shift gears, etc.) that the decision-making block determinesshould be performed by the vehicle. Based on the task selection, thedecision-making block activates one or more task blocks corresponding tothe task selection. The activated task blocks then execute predeterminedinstructions to control the actuation subsystem to actuate the vehicleand perform the selected tasks (e.g., depress the brake pedal ordownshift to slow down, rotate the steering wheel to change lanes,etc.).

In some variations and examples, control of the vehicle can betransferred between the vehicle control system, a teleoperator, and/or alocal driver (e.g., driver located within the vehicle). Transfers can beinitiated and/or triggered based on events. Events that can triggercontrol transfer include: measurement of a specific set or pattern ofvehicle parameter values associated with a vehicle action or behavior(e.g., accelerometer data indicating a swerve or skid), manualactivation of a mode-transfer switch (e.g., an operator hitting anemergency stop button), contextual events (e.g., location, such asproximity to a highway exit en route; time of day; the driver profile;whether the vehicle is on the highway; etc.), or any other suitabletrigger events. Trigger events can be determined from GPS and/or mapdata, images from an internally- or externally-facing camera and/oranalysis thereof, accelerometer and/or IMU data, and any other suitablesensor data and/or processed data. In a first example, the systemincludes an emergency-stop button attached to the steering wheel, whichgives full control of the vehicle to the local operator when depressed(e.g., actuatable control authority is removed from the decision-makingblock and/or the teleoperator). In another example, an emergency-stopfoot pedal is attached to the pedal actuation assembly of the actuationsubsystem, and depressing the emergency-stop pedal gives full control ofthe vehicle to the local operator and physically disengages the actuatorof the pedal actuation assembly from the vehicle control pedals. Inanother example, the steering actuation assembly automaticallydisengages upon detecting a threshold opposing torque (e.g., exerted bya local driver against the drive motor of the steering actuationassembly) and transfers control of the vehicle to the local operator. Inanother example, the system includes an autonomous ON/OFF switch thatcan be used by the teleoperator to switch control of the vehicle betweenthe teleoperator and the vehicle control system (e.g., operating in theautonomous driving mode). In another example, the system includes atoggle switch actuatable by the local operator to switch between in-seat(e.g., wherein the local operator is in the driver's seat of thevehicle) and out-of-seat (e.g., wherein the local operator is within thevehicle but not in the driver's seat) vehicle operation modes. However,control of the vehicle can be otherwise suitably transferred between anyother suitable operators and/or entities.

In another example of a portion of the method of use, the local operatorcan view a touch-sensitive display of the system that includes differentvisual indicators (e.g., color codes) displayed to the local operator(e.g., Green: ok, Yellow: cautious, and Red: takeover), wherein thesystem also includes an audio alert (e.g., a buzzer) that activatesprior to the changes in the visual indicator. In this example, there isa countdown rendered on the screen (e.g., a five second countdown) whenthe visual indicator has changed to indicate the local operator shouldtake command of the vehicle. After the countdown has completed, if thelocal operator has not assumed control of the vehicle, the vehiclecontrol system can control the vehicle to decelerate and pull to theshoulder at the first available opportunity.

The systems and methods of the preferred embodiment and variationsthereof can be embodied and/or implemented at least in part as a machineconfigured to receive a computer-readable medium storingcomputer-readable instructions. The instructions are preferably executedby computer-executable components preferably integrated with the systemand one or more portions of the processor and/or the controller 430. Thecomputer-readable medium can be stored on any suitable computer-readablemedia such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD orDVD), hard drives, floppy drives, or any suitable device. Thecomputer-executable component is preferably a general or applicationspecific processor, but any suitable dedicated hardware orhardware/firmware combination device can alternatively or additionallyexecute the instructions.

Although omitted for conciseness, the preferred embodiments includeevery combination and permutation of the various system componentsand/or method blocks.

The FIGURES illustrate the architecture, functionality and operation ofpossible implementations of systems, methods and computer programproducts according to preferred embodiments, example configurations, andvariations thereof. In this regard, each block in the flowchart or blockdiagrams may represent a module, segment, step, or portion of code,which comprises one or more executable instructions for implementing thespecified logical function(s). It should also be noted that, in somealternative implementations, the functions noted in the block can occurout of the order noted in the FIGURES. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the preferred embodiments of the invention withoutdeparting from the scope of this invention defined in the followingclaims.

What is claimed is:
 1. A system for controlling a vehicle navigating aroadway, comprising: a perception module that generates sensor dataindicative of traffic objects proximal the vehicle, wherein theperception module analyzes the traffic objects to generate a cost map ofthe roadway; a behavior planning module that generates plannerprimitives based on the cost map, wherein the behavior planning modulecomprises a decision-making block that comprises a probabilistic model;a local planning module comprising a set of task blocks, each of the setof task blocks comprising a deterministic set of rules, wherein thelocal planning module selects a task block based on the plannerprimitives, and uses the deterministic set of rules of the selected taskblock to generate control commands based on the cost map; and a controlmodule comprising an actuation subsystem, wherein the control modulereceives the control commands from the local planning module andcontrols the actuation subsystem based on the control commands.
 2. Thesystem of claim 1, wherein the sensor data comprises at least one of aset of positions and a set of trajectories, the set of positions and theset of trajectories associated with the traffic objects, wherein thetraffic objects are classified as at least one of a neighboring vehicleand a static traffic object by an object analysis block of theperception module that processes the sensor data.
 3. The system of claim1, wherein the perception module outputs a geographic location of thevehicle, and further comprising a mission planning module that receivesthe geographic location from the perception module and generates a routeplan based on the geographic location and a destination, wherein thebehavior planning module receives the route plan from the missionplanning module and generates the planner primitives based on the routeplan.
 4. The system of claim 1, wherein the cost map comprises atwo-dimensional mapping of a set of weights to an associated set ofpositions on the roadway.
 5. The system of claim 4, wherein each weightof the set of weights corresponds to a risk value of occurrence of anadverse event were the vehicle to be located at the associated position.6. The system of claim 1, further comprising a prediction block thatestimates future positions of traffic objects relative to the vehicle,based on computed trajectories of the traffic objects, wherein thebehavior module generates planner primitives based on the futurepositions of traffic objects.
 7. The system of claim 1, furthercomprising a finite-state automator that selects a subset of allowedplanner primitives from a set of planner primitives, based on the costmap.
 8. The system of claim 7, wherein the subset of allowed plannerprimitives excludes a lane-change primitive, based on sensor dataindicative that traffic objects are positioned proximal a side of thevehicle.
 9. The system of claim 1, further comprising a training modulethat receives the cost map and the sensor data from the perceptionmodule, receives driver input from a vehicle operator, and generates theprobabilistic model of the behavior planning module based on the driverinput, the cost map, and the sensor data.
 10. The system of claim 9,wherein the vehicle operator is a remote vehicle operator residingoutside the vehicle, further comprising a remote teleoperationinterface, comprising a display and a set of inputs, that renders thesensor data to the remote vehicle operator at the display and receivesthe driver input from the remote vehicle operator at the set of inputs,wherein the set of inputs comprises a steering wheel input, a gas pedalinput, a brake pedal input, and a transmission input.
 11. The system ofclaim 1, wherein each of the set of task blocks consists essentially ofthe deterministic set of rules.
 12. The system of claim 11, wherein thedeterministic set of rules comprises an explicitly-programmed set ofrules.
 13. The system of claim 1, wherein the probabilistic modelcomprises a trained machine-learning model.
 14. The system of claim 13,wherein the decision-making block consists essentially of the trainedmachine-learning model.
 15. A method for controlling a vehicle,comprising: continuously sampling, at a sensor subsystem of the vehicle,sensor data comprising an image stream, a localization signal, andoperational data; transmitting the image stream, the localizationsignal, and the operational data to a remote teleoperation interfaceassociated with a teleoperator; receiving, at a behavior planning moduleof the vehicle, a first directive from the teleoperator by way of theremote teleoperation interface, wherein the behavior planning modulecomprises a probabilistic model; generating a planner primitive at thebehavior planning module based on the directive using the probabilisticmodel; selecting, at a local planning module of the vehicle, a taskblock based on the planner primitive, wherein the task block comprises adeterministic set of rules; controlling the vehicle, at a control moduleof the vehicle, by executing the deterministic set of rules of theselected task block based on the sensor data; receiving a seconddirective from the teleoperator, in response to the vehicle entering ageographic region having a predetermined characteristic; transferringplanning authority to the behavior planning module of the vehicle, inresponse to receiving the second directive; automatically generating asecond planner primitive at the behavior planning module based on thesensor data; automatically selecting a second task block, wherein thesecond task block comprises a second deterministic set of rules, basedon the second planner primitive; controlling the vehicle by executingthe second deterministic set of rules of the second task block based onthe sensor data; and automatically transferring planning authority tothe teleoperator, in response to the vehicle reaching a predeterminedgeographic location.
 16. The method of claim 15, further comprisingrendering the operational data at the remote teleoperation interface.17. The method of claim 15, wherein the task block consists essentiallyof the deterministic set of rules.
 18. The method of claim 15, whereinthe probabilistic model comprises a trained machine-learning model. 19.The method of claim 18, further comprising training the trainedmachine-learning model based on the first directive in combination withthe sensor data.
 20. The method of claim 18, wherein the behaviorplanning module consists essentially of the trained machine-learningmodel.